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Abstract.  Task-Centered  Design  (TCD)  of  human-system  interfaces  focuses  on 
supporting  the  user  throughout  all  phases  of  tasks,  from  initiation  to  completion, 
TCD  typically  requires  software  that  monitors  aspects  of  system  information  to 
trigger  tasks,  develop  user-friendly  information  sets,  propose  task  solutions  and 
actions,  and  confirm  actions  as  directed  and  approved  by  the  operator.  The 
operator  monitors  tasks  awaiting  completion  on  a  Task  Manager  display.  We 
demonstrate  that  moment-to-moment  operator  workload  monitoring  is  greatly 
facilitated  by  TCD.  Workload  estimates  were  obtained  every  2-min  over  the 
course  of  a  35-min  test  session  during  an  air  defense  command  and  control 
scenario.  Workload  was  readily  modeled  by  the  task  loading,  and  the  density  of 
track  icons  on  the  display.  A  second  study  related  the  unitary  workload 
estimates  to  NASA  TLX  workload  subscales.  Unpublished  data  from  our 
laboratory  indicated  that  eye  activity  measures  (e.g„  blink  frequency  and 
duration,  pupil  diameter,  fixation  frequency  and  dwell  time)  did  not  improve  the 
estimation  of  workload.  These  findings  indicate  that  at  least  for  well-executed 
TCD  systems,  eye  tracking  technologies  may  be  best  employed  to  monitor  for 
fatigue  and  incongruities  between  the  focus  of  attention  and  task  requirements. 
Recent  findings  using  EEG  hold  promise  for  the  identification  of  specific  brain 
signatures  of  confusion,  orientation,  and  loss  of  situational  awareness.  Thus  the 
critical  element  of  human  directed  systems  is  good  initial  design.  Understanding 
of  the  task  will  lead  to  system  automation  that  can  balance  the  workload  of  the 
operator,  who  is  functioning  in  a  normal  state.  However,  physiological 
monitoring  will  be  most  useful  if  operators  veer  beyond  their  normal  conditions 
and  are  confused,  overloaded,  disoriented  or  have  other  impairments  to  their 
abilities.  By  detecting  the  operator’s  loss  of  function  early,  inappropriate 
operator  inputs  can  potentially  be  avoided. 


1  Introduction 

The  purpose  of  this  paper  is  twofold.  The  first  is  to  describe  how  using  appropriate 
design  strategies  can  improve  user  performance  and  significantly  mitigate  task 
overload  conditions.  The  second  purpose  is  to  describe  how  psychophysioiogical  and 
augmented  cognition  methods  can  best  be  employed  when  “task  management”  design 
principles  are  used.  There  has  been  a  plethora  of  studies  examining  how 
psychophysioiogical  variables  (e.g.,  electro-encephalogram,  eye  activity,  heart  rate 
and  variability)  change  as  a  function  of  task  workload  (see  Craven  et  al,,  2006; 
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Poythress  et  a].,  2006;  Wilson  &  Russell,  2003),  and  how  these  measures  might  be 
used  to  monitor  human  operators  for  task  overload  and/or  used  to  trigger  automated 
processes.  Using  a  Task-Centered  Design  approach,  where  an  operator’s  job  is  broken 
down  into  tasks,  and  subsequently  decomposed  into  sub-tasks,  provides  an 
opportunity  to  accurately  measure  moment-to-moment  workload  fluctuations  (see 
Campbell  et  al.,  2003;  Osga  &  Van  Orden,  2000).  Without  the  requirement  to  monitor 
for  workload,  the  application  of  operator  state  monitoring  technologies  can  focus  on 
issues  such  as  confusion,  stress,  drowsiness,  and  forms  of  loss  of  spatial  and 
situational  awareness.  Some  recent  findings  are  discussed  which  suggest  that  neural 
markers  for  some  of  the  aforementioned  cognitive  states  may  be  detected  when  they 
are  not  consciously  perceived. 

The  main  tenet  of  Task  Centered  Design  (TCD)  is  supporting  the  user  through  all 
task  phases.  This  involves  not  only  presenting  the  typical  “alert”  to  the  user,  but 
building  information  sets  specific  to  the  task,  recommending  a  course  of  action,  and 
confirming  that  the  task  was  completed  upon  approval  of  the  actions  by  the  system. 
For  example,  within  the  air  traffic  control  domain,  specific  tasks  might  be  triggered 
when  automation  detects  aircraft  within  distances  appropriate  to  begin  descent  for 
landing,  when  aircraft  are  on  courses  that  will  result  in  violated  spatial  separation,  or 
when  spacing  for  approach  controllers  could  be  improved.  In  such  instances,  a  task 
icon  could  appear  on  a  task  manager  display,  which  when  selected,  would  open  a  task 
information  set  window.  The  information  set  pertinent  for  the  task  would  contain 
background  information  and,  more  importantly,  a  recommended  course  of  action  for 
approval.  For  a  spacing  task,  the  recommended  course  of  action  might  be  to  reduce 
the  speed  of  a  particular  aircraft.  Upon  operator  approval  of  the  course  of  action,  the 
system  would  generate  an  electronic  voice  command  to  the  aircraft  (or  alternative 
data  link)  with  new  air  speed  guidance. 

Several  benefits  of  task  managed  design  include  an  active  (vice  passive)  control  of 
the  processes  by  the  user,  support  through  most  or  all  phases  of  the  task,  and  the 
potential  ability  of  automation  to  forecast  critical  events  and  time  tasks  so  that  they 
occur  sooner  than  would  otherwise.  In  other  words,  tasks  could  be  forecasted  by  the 
system  and  presented  to  the  user  prior  to  their  typical  trigger  points  as  potential 
backlogs  are  detected.  This  form  of  task  management  is  important,  as  it  is  well  know 
that  operators  have  a  poor  sense  of  time  management  under  high  workload  conditions. 


2  Task  Centered  Design  and  Workload  Assessment 

Van  Orden  (2001)  examined  how  workload  correlated  with  task  loading  and  other 
variables  in  a  40-minute  simulated  air  defense  task  hosted  upon  a  TCD  work  station. 
Tasks  included  reporting  new  air  contacts,  reporting  identification  changes  to  air 
contacts,  and  sending  messages  to  aircraft  entering  restricted  areas.  The  participants 
provided  subjective  workload  estimates  every  two  minutes,  and  these  were  matched 
with  the  number  of  tasks  appearing  on  a  task  manager  display,  and  the  number  of  air 
contacts  present  on  the  display.  Results  indicated  that  the  number  of  air  contacts 
present  on  the  display  combined  with  the  number  of  tasks  present  correlated 
significantly  with  the  subjective  workload  measure.  For  several  participants, 
psychomotor  activity  (mouse  clicks  to  reveal  information  about  specific  air  contacts) 
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was  also  included,  in  order  to  capture  work  associated  with  gaining  awareness  of  the 
air  contacts  on  the  display.  Data  for  one  such  participant  is  included  in  Figure  1.  (Not 
shown  are  number  of  air  contacts  on  the  display,  which  varied  between  15  and  30,  and 
increased  steadily  over  the  duration  of  each  experimental  session.).  Combined 
measures  of  contact  density,  number  of  tasks  to  be  performed,  and  psychomotor 
activity  accounted  for  70  percent  of  the  variance  in  subjective  workload  estimates  for 
this  participant.  Interpolating  subjective  workload  from  the  aforementioned  variables 
produced  a  moment-to-moment  workload  series,  shown  in  Figure  2. 


Fig.  1.  Plot  of  frequency  of  mouse  click  activity  (dashed  line),  task  load  (solid  line)  and 
subjective  workload  estimates  (filled  squares)  over  time  in  minutes  for  the  test  session 

The  preceding  description  on  deriving  a  moment-to-moment  workload  time  series 
from  data  easily  obtained  from  system  and  user  activity  measures  underscores  two 
points:  First,  that  task  data  from  a  TCD  implementation  is  an  important  factor  in  the 
determination  of  workload.  The  second  point  is  that  psychophysiological  data  would 
add  little  to  improving  the  workload  estimates.  However,  data  on  the  operator  state 
could  be  very  beneficial  if  applied  for  other  purposes.  Tt  has  been  previously 
demonstrated  that  psychophysiological  data  would  be  useful  for  the  determination  of 
drowsiness  (Van  Orden,  K.F.,  Jung,  T-P.,  &  Makeig,  2000;;  Makeig  &  Jung,  1996) 
and  loss  of  attentional  focus  (Makeig,  Jung  &  Sejnowski,  (2000).  Recent  findings 
suggest  that  brain  activity  and  electroencephalographic  (EEG)  measures  might 
provide  information  on  cognitive  states  such  as  false  memories  (Garoff-Eaton, 
Slotnick,  &  Schacter,  2006;  Schacter,  Reiman,  Curran  et  al„  1996)  or  spatial 
disorientation  (Viirre  et  al.,  2006).  These  findings  indicate  that  similar  measures 
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Fig.  2.  Subjective  workload  for  one  participant  based  upon  a  regression  model  using 
psychomotor  activity,  task  loading,  and  the  number  of  aircraft  on  the  display  as  input 

might  be  capable  of  identifying  such  operator  states  as  self-deception  or  loss  of 
situation  awareness;  conditions  beyond  the  normal  scope  of  performance  of  trained 
operators.  This  type  of  monitoring  could  serve  as  a  safe-guard,  or  used  as  a  training 
toot,  especially  under  grueling  or  stressful  operating  conditions,  or  in  advanced 
training  scenarios. 


3  Uncertainty,  Disorientation,  and  Self  Deception 

Schacter,  Reiman,  Curran  et  al.,  (1996)  demonstrated  that  positron  emission 
tomography  (PET)  recordings  differed  as  a  function  of  whether  participants  had 
accurately  or  falsely  remembered  words  from  a  previously  studied  list.  The  PET 
activation  differences  between  these  conditions  were  greatest  in  the  pre-frontal 
cortical  regions  of  the  brain.  There  were  clear  differences  between  correct  assertions 
and  incorrect  ones.  A  rudimentary  interpretation  of  this  result  was  that  in  the  incorrect 
assertion  state,  there  was  detectable  neural  activity  reflecting  uncertainty  in  the 
decision.  Thus,  as  the  subject  was  forced  to  make  an  assertion,  there  were  clear 
differences  between  assertions  that  were  made  that  were  accurate  and  those  that  were 
wrong.  The  disconcordance  of  brain  activation  from  conscious  actions  and  underlying 
unconscious  decision  making  has  far  reaching  ramifications.  An  important  practical 
application  is  that  operator  confusion  or  uncertainty  can  likely  be  detected.  Thus, 
situations  where  an  operator  implements  an  input,  but  is  uncertain  about  it,  the 
operator,  and  their  human  or  automated  supervisors  can  be  alerted  to  that  state  and 
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provide  further  analysis.  For  example,  if  an  operator  is  required  to  make  an  input,  but 
has  behavioral  or  neural  markers  that  show  uncertainty,  the  system  or  an  instructor 
could  respond  to  it  and  re-run  the  scenario  or  re-present  the  data  to  enable  the  operator 
to  choose  again. 

Example:  Three  Dimensional  (3D)  Navigation 

Given  the  ability  to  monitor  neural  activity  related  to  decision  making  and  memory 
assertions,  an  era  where  it  will  be  possible  to  monitor  operators  during  specific 
cognitive  tasks  is  now  upon  us.  Thus  TCD  will  become  more  rigorous  and  instead  of 
using  global  non-specific  concepts  such  as  “work  load”,  we  will  be  able  to  implement 
monitoring  of  conditions  such  as  memory  and  decision  processing. 


Fig.  3.  Three  Dimensional  scene  used  during  visual  tracking  task 


Data  from  neural  monitoring  during  a  navigation  task  suggests  an  example.  In 
Strychacz  et.  A1  (2005),  we  had  subjects  carry  out  a  tracking  task  while  virtually 
traveling  through  a  three-dimensional  scene  (See  figure  below).  We  then  had  subjects 
view  another  3  dimensional  scene  that  resulted  in  the  onset  of  motion  sickness. 
Monitoring  neural  patterns  using  high  density  EEG  demonstrated  a  strong  signature  of 
Alpha  wave  and  alpha  harmonic  peaks  in  the  posterior  mid-parietal  region  (See  Figure 
X  below).  These  peaks  were  eliminated  when  the  motion  sickness  appeared. 

Further,  we  found  similar  neural  activity  changes  in  the  same  region  during  a 
navigation  task  where  we  implemented  Schacter’s  paradigm  (Viirre,  et.  al.  2006). 
Participants  were  asked  to  observe  a  rectilinear  virtual  environment  on  a  computer 
screen  (See  Figure  3).  The  point  of  view  of  observer  was  randomly  shifted  and  rotated 
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Fig.  4.  Prominent  power  peaks  at  20  and  40  Hz  in  ventral  midline  sources  during  the  tracking 
(light  colored  line)  that  are  suppressed  during  motion  sickness  (darker  line) 

and  the  subject  was  asked  to  keep  track  of  their  apparent  compass  heading. 
Randomly,  the  scene  movement  would  stop  and  the  subject  was  presented  with  a 
compass  rose  and  a  “lost  button”.  They  indicated  which  heading  they  believed  they 
were  facing  or,  if  they  were  not  confident  of  the  heading,  they  pushed  the  “lost” 
button.  Continuous  EEG  and  eye  movement  data  were  recorded  during  the  trials  on 
128  scalp  and  2  eye  channels.  EEG  activities  were  labeled  “correct”  (heading  fashion 
to  the  data  presented  above  in  Figure  4.  Similar  to  word  recall  tasks,  frontal  midline 
sources  were  found  to  show  significant  differences  between  correct  and  incorrect  in 
the  interval  500  ms  after  the  button  press,  indicated  within  +/-  45  degrees),  “incorrect” 
or  “lost”.  Subjects  were  found  to  make  incorrect  heading  assertions  approximately  20- 
30%  of  the  time,  even  though  they  were  given  the  option  to  indicate  they  were  lost. 
Significant  differences  between  “correct”  and  “incorrect”  responses  were  apparent  in 
neural  activity  in  ventral  posterior  midline  sources  in  all  subjects  in  the  time  interval 
500  ms  prior  to  the  button  response.  These  differences  occurred  in  the  alpha  EEG 
frequency  range  in  similar. 

These  results  suggest  that  it  may  be  possible  to  detect  in  near  real-time  neural 
activity  markers  of  operator  task  attention  to  a  three  dimensional  task  and  to  the  loss 
of  that  attention.  Thus  when  training  pilots  to  perform  instrument  maneuvers  or  when 
training  radiologists  to  interpret  moving  three-dimensional  images  such  as  ultra 
sound,  an  automated  augmented  cognition  system  or  an  instructor  would  be  able  to 
detect  when  the  operator  was  not  on  task.  This  information  could  be  used  to  enhance 
task  performance  by  notifying  the  trainee  pilots  to  re-orient  using  navigation 
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instruments;  or  provided  feedback  to  trainee  radioiogists  to  re-establish  their 
orientation. 

4  Conclusion 

Task  Centered  Design  is  an  important  process  in  the  Human  Factors  component  of 
systems  development.  The  combination  of  TCD  and  augmented  cognition 
technologies  -  used  to  detect  neurophysiological  state  —  is  exciting  and  offers  the 
potential  to  improve  human-machine  interactions.  However,  significant  challenges 
remain  with  regard  to  how  an  operator’s  state  is  monitored,  and  different  approaches 
should  be  carefully  examined  in  the  earliest  phases  of  system  design.  Careful  task- 
centered  design  review  of  a  system  will  yield  more  benefit  than  grafting 
neurophysiological  state  assessment  onto  an  existing  interface.  TCD  should  yield  a 
thorough  understanding  of  the  vast  majority  of  system  states  that  might  arise,  and  then 
reveal  opportunities  to  appropriately  manage  them. 

Neurophysiological  state  assessment  is  best  used  to  detect  operators  who  are  “out- 
of-range”  for  normal  operator  conditions:  such  as  -  overstressed,  over-loaded, 
disoriented  or  other  situations.  Neurophysiological  assessment  is  becoming  refined  in 
die  ability  to  detect  specific  state  conditions  in  a  variety  of  real-world  environments. 
Instead  of  a  pilot  being  in  a  state  of  “high  workload”  (which  the  computer  could 
already  determine),  state  assessment  could  provide  more  detailed  information  such  as; 
they  are  not  correctly  monitoring  their  three  dimensional  orientation.  Such  state 
assessments  would  provide  greater  opportunity  for  detecting  and  mitigating  human- 
system  failures. 

As  the  technological  advances  occur,  the  difficulty  in  using  these  technologies  will 
become  designing  paradigms  for  identifying  the  appropriate  neural  signals.  For 
example,  the  operational  aviation  environment  is  vastly  different  from  a  laboratory 
setting.  Two  technologies  that  may  offer  some  promise  for  real-time  detection  during 
operations  on  moving  platforms:  advanced  electro-encephalography  (EEG)  and 
functional  near  infrared  (fNIR)  recording.  EEG  systems  now  incorporate  miniature 
amplifiers  at  the  electrode  that  greatly  enhance  signal  to  noise  and  reject  local 
interference  sources.  Portable  wireless  EEG  sets  now  exist.  FNIR  is  a  newer 
technology  that  can  measure  regional  blood  oxygenation  changes  in  the  outer  layers 
of  the  cerebral  cortex  using  infrared  light.  EEG  and  fNIR  have  been  used  to  look  for 
signatures  related  to  GLOC  and  related  phenomena.  It  provides  reliably  repeatable 
measures  that  are  empirically  useable  if  not  perfectly  correlated  to  known  physiologic 
changes.  Further,  limited  studies  relevant  to  target  detection  and  loss  of  SA  can  be 
carried  out  in  functional  magnetic  resonance  imaging  (fMRI)  systems.  FMRI  can  be 
used  with  some  simulations  of  disorienting  conditions,  but  can  not  be  applied  to 
operational  conditions.  Fortunately,  with  its  fine  spatial  resolution,  it  can  greatly  assist 
in  identification  of  specific  brain  areas  involved  in  motion  processing  and  thus  be 
used  to  compare  data  across  subjects  and  further  refine  operational  tools  and 


measures. 
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